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Executive Summary 



Launch of the Student Achievement Initiative 

In September 2007, the Washington State Board for Community and Technical Colleges 
(SBCTC) officially launched the Student Achievement Initiative, a system- wide policy to reward 
colleges for improvements in student achievement. Developed by a task force comprised of State 
Board members, college trustees, presidents, and faculty representatives, the policy emphasizes 
three overarching principles: 1) the initiative should lead to improved educational attainments for 
students and, in particular, should be aimed at boosting those educational attainments shown by 
research to be correlated with the earning of higher future wages by students; 2) the initiative 
should allow colleges sufficient flexibility to improve student achievement according to their 
local needs; 3) the initiative should result in the identification and implementation of successful 
practices to improve student achievement system-wide. 

Under the initiative, Washington’s community and technical colleges will receive financial 
rewards for increasing the rate at which they accrue “achievement points.” Achievement points 
are generated when students achieve key attainments in four categories. Colleges each points 
when students 1) achieve gains in adult basic skills and pre-college remedial courses; 2) 
complete a college-level math course; 3) earn college credits; or 4) complete a certificate, degree, 
or apprenticeship training program. 

The SBCTC staff chose the 2006-07 academic year as the baseline year for measuring the 
performance of colleges using the achievement point categories. The first “performance year” 
will be the 2008-09 academic year. In that year and in subsequent years, the performance of 
colleges will be assessed by comparing the total number of achievement points they generate in a 
given year with their baseline year point totals. The State Board is requesting $7 million from the 
legislature for the initiative during the 2009- 1 1 biennium. 

The State Board designated the 2007-08 academic year as the initiative’s “learning year.” The 
learning year was designed to enable colleges to better understand their performance across the 
achievement point categories and to begin developing strategies for improving student 
attainments in achievement point categories. Each of the state’s 34 community and technical 
colleges received an allocation of approximately $51,000 in fall 2007 as start-up funding. 
Throughout the learning year, the State Board staff sought to raise awareness and understanding 
of the initiative, help colleges learn how to use data on their achievement point performance to 
identify areas for improvement, and promote sharing of student success strategies across 
campuses. 
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The Initiative in Comparative Context 

Performance funding assumes that financial rewards will serve to change institutional behavior. 
By choosing measures to gauge college performance, policymakers can both motivate colleges to 
improve their performance and encourage them to prioritize their efforts according to particular 
goals or outcomes. 

In 1978 Tennessee was the first state to adopt a set of performance measures that specifically 
provided higher education institutions with financial incentives for improving student outcomes. 
Higher education performance funding reached peak popularity in 2001, when 19 states had 
implemented such policies. Several other states have experimented with performance funding 
over the years, but most now have performance reporting without attaching funds. It should be 
noted that more than a decade ago Washington State implemented a short-lived performance 
funding system for the postsecondary sector, which had minimal input or buy-in from the state’s 
colleges and universities. 

Although the present initiative for Washington State community and technical college shares 
some important characteristics with earlier performance funding efforts that have taken place 
across the country, it also has some distinctive features. These include the following: 

• Rather than using a set of performance measures dictated by legislation or external 
policymakers, the initiative’s measures were recommended by a diverse task force of 
college representatives and were informed by research (conducted by the SBCTC and the 
Community College Research Center) on student progression that sought to identify key 
educational attainments associated with the increased probability of college and labor 
market success. 

• By rewarding colleges for increasing the rate of intermediate educational attainments (in 
addition to completing credentials) in a broad continuum of mission areas, including 
adult basic skills and college remedial education, the initiative seeks to focus colleges’ 
attention on the progression of all students, regardless of where they begin. 

• Unlike measures such as job placement rates, which are dependent on the economy and 
other factors, the task force chose performance measures that were more likely to be 
within a college’s control. 

• The achievement point data supplied to the colleges is intended to enable them not only 
to measure their performance, but to identify areas for improvement and success and to 
evaluate the effectiveness of efforts made to improve student success. 

• Colleges are not compared with one another to earn financial rewards, but rather are 
measured against their own historical performance. 

The Washington State Board’s efforts to design a set of student- focused performance indicators 
for community colleges has generated substantial interest from other states. The success or 
failure of the Student Achievement Initiative to support improved student outcomes will be 
watched closely. 
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Formative Evaluation: Scope and Methods 



Researchers from the Community College Research Center (CCRC) at Teachers College, 
Columbia University, conducted an independent qualitative review of the Student Achievement 
Initiative during the 2007-08 learning year. The purpose of this formative evaluation was to 
assess the extent of awareness and understanding of the initiative among college personnel, 
examine the initial responses to it by the colleges, and identify opportunities for and potential 
barriers to the further development of the initiative. 

The CCRC research team conducted extensive interviews with college personnel through site 
visits to eight colleges during spring 2008 and conducted phone interviews with college 
personnel at an additional nine colleges through September 2008. The research team interviewed 
college trustees, presidents, vice presidents of instruction and student services, institutional 
research personnel, faculty, and student services staff. In all, we interviewed over 240 
individuals connected with the 17 colleges. 

A second round of in-person and telephone interviews was conducted by CCRC in early fall 
2008 with key internal and external stakeholders and with policymakers to better understand the 
policy context within which the Student Achievement Initiative was conceived and is now being 
implemented. Among those interviewed were members of the State Board, executive staff from 
the SBCTC, staff and board members from the Washington Higher Education Coordinating 
Board, a senior analyst from the governor’s Government Management Accountability and 
Performance (GMAP) office, and a state senator who has been a leader in issues related to higher 
education accountability. 



Formative Evaluation: Summary of Findings 

• Colleges strongly supported the initiative’s goals and the principles of the achievement 
point framework 

Among those we interviewed at 17 of the state’s 34 community and technical colleges, there was 
widespread support for the initiative’s goal of focusing attention and resources on helping all 
students, including those who start in basic skills and college remedial programs, to successfully 
progress through their educational programs. There was also strong support for the focus on 
increasing student progression across intermediate levels of achievement. The faculty, student 
support staff, and administrators we interviewed generally agreed that this incremental 
progression framework is much more relevant to their work with students than accountability 
measures the state has used in the past, which were based primarily on final outcomes. Virtually 
everyone who was familiar with the research the State Board staff conducted to inform the 
development of the overall framework and the specific performance measures had high regard 
for it. 
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• Awareness of the initiative was limited among the colleges’ rank and file 

While State Board staff did much to try to build awareness of the initiative among the colleges, 
at the colleges in this study familiarity with the initiative is mostly limited to college leadership, 
senior administrators, and a small number of college personnel involved in initiative efforts at 
the colleges. 

Several colleges were waiting for a better understanding of their performance on the complete 
2007-08 data before expanding communication about the initiative to faculty and staff. Senior 
administrators at several colleges said that, having had to respond to numerous State Board 
initiatives in recent years, they are reluctant to engage campus-wide constituencies in another 
external directive until they are assured it is likely to gain support from the state legislature. 

The lack of awareness and clarity about the initiative should not be too surprising, however. The 
initiative was developed and is being implemented on a relatively short timeline, and the colleges 
were still in the first “learning year” of the initiative during our interviews. Still, college 
personnel at several colleges suggested that clear and transparent communication about the 
initiative would allay some of the uncertainty and anxiety surrounding the effort and help build 
support for it. 

• Colleges grappled with their performance data throughout the learning year 

There was considerable uncertainty among the colleges about exactly how student progression is 
measured using the achievement point framework. Most colleges struggled throughout the 
learning year to understand the details of the performance measures. At the time of our 
interviews, most colleges were still in the process of understanding how their baseline year 
achievement point totals were calculated. The lack of institutional research capacity hampered 
several colleges’ ability to analyze the data for a more thorough understanding of their college’s 
performance. Even colleges with greater IR capacity were having difficulty making sense of the 
data and figuring out how to use the information to monitor student progression and evaluate 
efforts to improve student outcomes. 

• Most colleges had not yet used an analysis of their performance data to plan new 
strategies to improve student achievement 

Because of these difficulties with the achievement point data, most colleges had not begun to use 
the Student Achievement Initiative database to develop strategies for increasing student success. 
Only a handful was planning new strategies to improve their achievement point performance. 
Most were relying instead on student success activities previously in place at their colleges. 
While college personnel were optimistic that these existing activities would help to increase 
student attainment of achievement points, most such efforts were small in scale. College leaders 
have not yet engaged faculty and staff to figure out what changes in policy and practices are 
needed to increase the rate at which students attain achievement points on a substantial scale. 



CCRC Formative Evaluation of the Student Achievement Initiative “Learning Year 



4 




• At many colleges, student services staff led efforts in response to the initiative 

Many interviewees we spoke to could not, or at least did not, distinguish between ongoing efforts 
to support student retention and success at their college and activities planned in response to the 
Student Achievement Initiative. At least half of the colleges in our study gave primary 
responsibility for responding to the initiative to their student services division. One reason is that 
at most of the colleges where we conducted interviews, student services divisions have primary 
responsibility for efforts to improve student retention and success, so it made sense for them to 
take the lead on this new effort. Student services personnel were enthusiastic about what they 
saw as an increased emphasis on student persistence by their colleges. Most indicated that their 
existing student retention efforts would help to improve their college’s performance on the 
achievement point measures of the Student Achievement Initiative. 

• Several colleges focused their initial efforts on basic skills and developmental 
education 

Several colleges focused their initial work on the initiative on basic skills and developmental 
education. There were several reasons for this. First, actions of the SBCTC staff responsible for 
basic skills helped raise awareness of the initiative among basic skills programs directors. 

Second, there was a widespread perception that colleges could earn more points in the basic 
skills achievement category than in those related to passing college-level math, completing 
college credits, or earning credentials. Third, most of the colleges had begun efforts to improve 
outcomes in basic skills and developmental education before the advent of the Student 
Achievement Initiative. Interviewees said that the initiative provided additional incentive to 
strengthen those efforts. College administrators said that there has been a growing recognition in 
recent years that too few students progress through the various levels of basic skills or 
developmental education to advance to college-level coursework. And finally, persons involved 
with basic skills programs were often perceived to be more receptive to the initiative because 
they are accustomed to being held accountable for student performance. Basic skills directors 
and instructors at several colleges suggested that existing state and federal performance 
requirements had given them experience in tracking the progress and outcomes of their students. 

• Some colleges were planning to implement measures with a likely one-time effect 

Some colleges were planning to implement measures that would likely have a one-time effect on 
increasing achievement point attainments by their students. Among the approaches proposed 
were these: conducting “degree audits” or transcript reviews to identify students who achieved 
the required credits for a certificate or degree but never received one; eliminating certificate or 
diploma fees and creating academic certificate programs; documenting more rigorously CASAS 
testing results for basic skills students; and increasing credit requirements for certain shorter 
occupational certificate programs so they will qualify for achievement points. 

• Several colleges were beginning to link the initiative to strategic planning and 
accreditation activities. 

Some college administrators were beginning to align the initiative’s performance measures with 
their strategic planning and re-accreditation activities. Six colleges had begun early efforts to 
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incorporate the initiative into their internal strategic planning processes. College personnel at at 
least eight colleges discussed how the initiative’s focus on documented student progression 
supported their institution’s re- accreditation efforts. 

• Colleges were concerned that use of the achievement point framework and the 
incentive funding model may produce unintended effects and place some colleges at a 
disadvantage 

While there was general support for the principles of the initiative’s achievement point 
framework and the model for rewarding achievement point gains, many interviewees voiced a 
number of concerns. Some interviewees felt that, in practice, their use might unintentionally alter 
their college’s priorities. For example, some faculty and administrators from career-technical and 
academic transfer programs expressed concern that the initiative’s heavy focus on basic .skills 
and lack of performance measures for job placements and baccalaureate transfers could draw 
resources away from their areas. 

Interviewees frequently suggested that use of the framework and the incentive model might 
disadvantage certain kinds of colleges, including 1) institutions with a strong academic transfer 
mission, because the potential for generating the most achievement points is found in the basic 
skills area; 2) institutions without capacity for significant enrollment growth, because colleges 
generally might be able to accrue achievement points by increasing enrollments rather than by 
improving rates of student progression; 3) high performing colleges, which may have less room 
to improve; and 4) institutions serving a high proportion of disadvantaged students, because it 
would be difficult for such institutions, who need to provide lots of costly wrap-around services, 
to earn as many points as colleges with easier- to-serve students. 

Some colleges were also anxious about how their performance would compare to other colleges 
(apart from the issue of funding). Concerns were also raised about the potential for “gaming” the 
system (by, for example, focusing efforts on increased enrollment rather than improved student 
progression). Interviewees at five colleges noted that, in practice, incentive funding could lead to 
a lowering of academic standards, and a few interviewees felt that its use could discourage 
colleges from sharing effective practices. 

• College presidents emphasized that, in order to be effective in improving college 
performance, the initiative must bring new funding to colleges, over and above base 
budget funding 

The potential of significant funding tied to institutional performance has generated concern and 
anxiety across the colleges, particularly among presidents. Most presidents we interviewed seem 
to agree that the amount currently being sought by the State Board from the legislature — $7 
million for the biennium, or about $100,000 per year per college — is large enough to motivate 
colleges to make changes in the way they operate, particularly because the funds would be added 
to their base budgets. Presidents we interviewed were unanimous in the view that to be effective 
in improving college performance, the initiative must bring new funding to the table over and 
above base budget funding. College leaders feared a situation where there would be no new 
funds to support performance funding and that colleges would instead be forced to earn some 
portion of their base budget funding based on the achievement point measures. Presidents argued 
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that this would lead to unhealthy competition among the colleges and would significantly reduce 
support for the initiative from the colleges. Although there were exceptions, most state-level 
observers we interviewed seemed to agree that funding above and beyond the base budget 
funding is essential for the initiative to be effective and sustainable. 

• While state policymakers indicated strong support for the initiative’s model of 
performance accountability and improvement, there is limited awareness of the 
initiative and no strong champions for it among state legislators. The looming fiscal 
crisis further threatens the SBCTC’s legislative request for new funding to support the 
initiative. 

The state-level stakeholders we interviewed were very enthusiastic about the initiative’s 
performance measurement model. Indeed, there are strong synergies between the initiative and 
the governor’s push for more outcome-oriented and accountable state programs. Because 
accountability pressures across state agencies are not likely to abate, state-level policymakers felt 
that the Student Achievement Initiative could give the community and technical colleges an 
advantage as they compete with other sectors for scare state resources. 

The Higher Education Coordinating Board seems committed to advocating with the legislature 
for the SBCTC’s request for new funding to support the initiative. However, state-level 
stakeholders indicated that the initiative is not well known and lacks strong champions among 
legislators. 

The State Board’s request to the legislature is threatened by the looming fiscal crisis facing 
Washington State and the country more generally. A decision by the legislature not to fully fund 
the initiative could affect future college support. At this point, however, the State Board seems 
intent on moving ahead with the request. 

Conclusion 

The State Board developed a performance measurement system as a practical means of 
encouraging Washington State community and technical colleges to improve their outcomes over 
time toward system goals and demonstrating the efficient use of public resources with more 
attention paid to data and accountability. The Board realized that buy-in from the colleges 
themselves was essential to such a plan. 

The evaluation we carried out during the initiative’s “learning year” shows that there is strong 
support among the colleges for the initiative’s goals and for the principles behind the initiative’s 
performance measurement model. Yet awareness of and knowledge about the initiative is still 
weak among many of the rank and file at most colleges. This should not be too surprising, 
however, given that the initiative is new and still evolving. Likewise, the problems that colleges 
are experiencing in understanding and using the performance data from the initiative database 
are not necessarily unexpected. Nevertheless, the need for greater clarity about the database will 
have to be addressed before the data will be disseminated to wider audiences within colleges and 
used extensively to identify strategies for improving student success. 
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While the colleges show general support for the initiative, they are concerned that in practice it 
may produce unintended consequences in the way that colleges respond to it. Furthermore, there 
is a general concern that some colleges may be placed at a disadvantage by the achievement 
point measures. And college presidents in particular are concerned that funds that colleges will 
receive for improved performance may be taken from their base budget funding. 

This fall the State Board plans to move forward with its request to the legislature for new 
funding for the initiative, but the Student Achievement Initiative is not well-known among state 
legislators. What is more, the state faces a looming budget crisis. This increases the importance 
for the State Board to develop a clear and compelling message and a strategic communications 
plan to educate policy makers about the initiative and its potential benefits for students and the 
state. 

While legislators may not support the initiative at the level requested by the State Board, they 
might be willing to provide more modest funding and allow the Board to extend the learning 
period for the initiative through the biennium as the state weathers the economic downturn. 
Given the fact that colleges are just beginning to analyze their data and consider changes in 
practices that would improve student achievement, extending the learning period would give the 
Board and the colleges a chance to deepen awareness and support of the initiative among faculty 
and staff, use the data to identify areas of weakness, and implement and evaluate strategies for 
improving student achievement. This would also increase the opportunity to examine to what 
extent and in what ways colleges change their practices in response to the initiative. Finally, 
given the widespread interest in the initiative by other states and funders nationally, the State 
Board may well be able to raise private foundation funds to supplement state funding of the 
initiative during such a period of further experimentation and evaluation. 
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Chapter 1 
Introduction 



A. The Student Achievement Initiative 

In 2006, the Washington State Board for Community and Technical Colleges (SBCTC) directed 
its staff to develop a proposal for a system- wide policy to reward colleges for improvements in 
student achievement. In response, the SBCTC staff convened a policy task force comprised of 
State Board members, college trustees, presidents, and faculty representatives. The task force 
was charged with developing the principles and goals of the policy and for recommending a 
design to the State Board. The task force solicited feedback from college personnel at the 34 
community and technical colleges across the state and received advice from national higher 
education experts on measuring student achievement, designing funding incentives, and 
sustaining performance funding. A system- wide advisory committee comprised of 
representatives from each of the community and technical college commissions and councils 
assisted in the design of the performance measures and provided input to the task force. 

In September 2007, the SBCTC officially adopted the task force’s proposal and launched the 
Student Achievement Initiative (Resolution 07-09-29). The policy outlined in the task force’s 
proposal and adopted by the State Board included three overarching principles: 

• The initiative should lead to improved educational attainment for students; the specific 
goal should be for students to reach the “tipping point” ^ and beyond. 

• The initiative should allow colleges sufficient flexibility to improve student achievement 
according to their local needs. 

• The initiative should result in the identification and implementation of successful 
practices to improve student achievement system-wide. 

Under the new initiative, Washington’s community and technical colleges will receive financial 
rewards for increasing the rate at which their students attain “achievement points”^ in four 
categories. Colleges accrue points when students: 1) achieve gains in adult basic skills and pre- 
college remedial courses; 2) complete a college-level math course; 3) earn college credits; and 4) 
complete a certificate, degree, or apprenticeship training program.^ 

The SBCTC staff chose the 2006-07 academic year as the baseline year for measuring the 
performance of colleges using the achievement point categories. The first “performance year” 



* This term refers to a 2005 State Board study that found that, five years after they first enrolled, community college 
students who reached the “tipping point” of at least two semesters of credits and a credential earned had 
substantially higher earnings than did students who did not make it that far. David Prince of the State Board and 
Davis Jenkins of the CCRC co-authored the study, which can be downloaded at: 
http://ccrc. tc.columbia.edu/Publication. asp?uid=288 . 

^ The SBCTC staff changed the terminology from “momentum points” to “achievement points” in spring 2008. 

^ The Community College Research Center (CCRC) at Teachers College, Columbia University assisted the State 
Board research staff to empirically test the achievement point measures. 
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will be the 2008-09 academic year. In that year and in subsequent years, the performance of 
colleges will be assessed by comparing the total number of achievement points they generate in a 
given year with their baseline year point totals. The State Board set aside $500,000 to be 
allocated to colleges in October 2009 for performance improvements in the 2008-09 academic 
year, with the funds to become part of the colleges’ base budgets. The State Board is requesting 
$7 million from the legislature for the initiative during the 2009- 1 1 biennium. 

The State Board designated the 2007-08 academic year as a “learning year.” The learning year 
was designed to enable colleges to better understand their performance across the achievement 
point categories and begin developing strategies for improving student attainment of 
achievement points. Each of the state’s 34 community and technical colleges received an 
allocation of approximately $51,000 in fall 2007 as start-up funding. Throughout the learning 
year, the State Board staff sought to raise awareness and understanding of the initiative, help 
colleges learn how to use data on their achievement point performance to identify areas for 
improvement, and promote sharing of student success strategies across campuses. 



B. Formative Evaluation of the Student Achievement Initiative “Learning Year” 

This report presents findings from an independent qualitative review of the Student Achievement 
Initiative conducted by the Community College Research Center (CCRC) at Teachers College, 
Columbia University, during the 2007-08 learning year."^ The purpose of this formative 
evaluation was to assess the extent of awareness and understanding of the initiative among 
college personnel, examine the initial responses to it by the colleges, and identify opportunities 
for and potential barriers to the further development of the initiative. Funding for this work was 
provided by College Spark Washington and by the Ford Foundation as part of the Bridges to 
Opportunity initiative and Fumina Foundation for Education as part of Achieving the Dream: 
Community Colleges Count. 

The CCRC research team conducted extensive interviews with college personnel through site 
visits to eight colleges during spring 2008 and conducted phone interviews with college 
personnel at an additional nine colleges through September 2008. These colleges were chosen to 
provide balanced representation of Washington State community and technical colleges in terms 
of size, urbanicity, and region of the state. The interviews followed a semi-structured protocol to 
ensure that similar questions were asked at every college. The team interviewed persons in 
comparable positions at the colleges, including college trustees, presidents, vice presidents of 
instruction and student services, institutional research personnel, faculty, and student services 
staff. On-site and telephone interviews were conducted either individually or in small groups and 
lasted approximately one hour. In all, we interviewed over 240 individuals connected with the 17 
colleges. Respondents were assured that their comments would be kept confidential. 

A second round of in-person and telephone interviews was conducted by CCRC in early fall 
2008 with key internal and external stakeholders and with policymakers to better understand the 
policy context within which the Student Achievement Initiative was conceived and is now being 



4 

CCRC is also planning to conduct a quantitative analysis of the initiative’s impact and effectiveness over the next 
four years. 
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implemented. Among those interviewed were members of the State Board, executive staff from 
the SBCTC, staff and board members from the Washington Higher Education Coordinating 
Board, a senior analyst from the governor’s Government Management Accountability and 
Performance (GMAP) office, and a state senator who has been a leader in issues related to higher 
education accountability. 



C. Organization of the Report 

This report presents findings and themes that have emerged through an analysis of data from the 
interviews with college personnel, policymakers, and other stakeholders. The report is organized 
as follows. Chapter 2 describes the policy context in which the Student Achievement Initiative 
was conceived and examines how the initiative is different from higher education performance 
funding schemes adopted in other states. Chapter 3 describes how college personnel were 
informed about the initiative, their level of understanding of it, and what they saw as its origins 
and purposes. Chapter 4 examines how the colleges organized their early efforts to implement 
the initiative during the learning year. Chapter 5 examines the views of college personnel on the 
initiative’s performance measurement model. Chapter 6 examines how college personnel viewed 
the initiative’s approach to incentive funding. Chapter 7 describes the views of higher education 
policymakers on the initiative as a whole. Chapter 8 presents our conclusions based on the 
findings and an assessment of the prospects for the future development of the initiative. 
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Chapter 2 

Background: The Policy Context for the Initiative 



This chapter explores the poliey eontext in whieh the Student Aehievement Initiative was 
conceived. 



A. Performance Funding: A Tool to Ensure Responsiveness to State Priorities 

Nationwide, state higher edueation polieymakers eoncerned with improving educational 
outcomes for students are usually eager to find ways to hold eolleges and universities 
aeeountable for student sueeess and improving their performanee over time. Yet, whether eleeted 
legislators or appointed state board members, polieymakers have few levers they ean use to 
influence institutional behaviors. Regulation and finanee are the main tools they have at their 
disposal to shape how publie eolleges and universities respond to pressing state priorities. 

Linking institutional funding to poliey goals through performanee funding is one speeifie way 
that policymakers have sought to induee eolleges to improve outcomes. 

Performanee funding assumes that finaneial rewards will serve to ehange institutional behavior. 
By ehoosing measures to gauge eollege performanee, polieymakers ean both motivate eolleges to 
improve their performanee and encourage them to prioritize their efforts according to particular 
goals or outcomes. 

In 1978 Tennessee was the first state to adopt a set of performanee measures that speeifieally 
provided higher edueation institutions with finaneial ineentives for improving student outeomes. 
Higher edueation performanee funding reaehed peak popularity in 2001, when 19 states had 
implemented sueh polieies. Several other states have experimented with performanee funding 
over the years, but most now have performanee reporting without attaehing funds. 

A review of the researeh literature reveals the following lessons learned from states that have 
experimented with performanee funding for eolleges and universities. 

• Performanee measures should reflect state priorities; at the same time, they should be 
designed to aeeommodate the multiple missions of higher edueation institutions. 

• Performanee funding should be suffieient to eneourage institutions to improve, but not so 
mueh that it leads to budget instability. 

• Funding should be stable and predietable — frequently ehanging funding levels hampers 
eolleges’ ability to do budget planning and ean erode their support over time. 

• Colleges prefer that performanee funding come from “new money” over and above base 
budget funds; yet a separate performanee fund budget item is suseeptible to outs when 
state budgets are tight. 
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• States should avoid a “zero-sum game” situation where colleges compete with one 
another for a fixed amount of funds and where one college can lose funding to others. 

• Colleges should be allowed to use performance funds as they see fit to improve student 
outcomes; putting “strings” on such funding can have unexpected consequences. 

• Higher education policymakers should involve educators in defining performance 
measures. 

• Performance incentive programs should be routinely reviewed to ensure that success is 
continually redefined as the state’s needs change. 

Research has thus far failed to provide definitive evidence that performance funding actually 
leads to improved institutional performance and improved student outcomes. One reason for this 
is that it is difficult to isolate the effects of specific policies on the performance of large and 
complex systems. Another reason is that in many states short attention spans on the part of a 
changing political leadership result in performance funding policies that are not in place long 
enough to demonstrate any lasting effects. 



B. Past Experience with Performance Funding in Washington 

The current Student Achievement Initiative is not the first effort on the part of Washington State 
policymakers to tie institutional funding more closely to outcomes. More than a decade ago a 
postsecondary performance funding system was championed by the Republican chair of the 
House Appropriations Committee in the legislature. According to higher education leaders who 
were in key roles at the time, the system was punitive, with institutions facing one percent budget 
cuts if they did not meet performance measures developed with minimal input or buy-in from the 
state’s colleges and universities. Republicans lost control of the legislature in the next election, 
and when the new Democratic leadership showed no interest, performance funding was 
abandoned. 

The demise of that earlier performance funding initiative did not mean that policymakers in the 
state had abandoned their interests in accountability, however. Upon her election in 2004, 
Washington Governor Chris Gregoire established the Government Management Accountability 
and Performance (GMAP) program. According to its website,^ the office was created “because 
the governor believes ‘It’s not enough just to set priorities.’ Every state agency, every program, 
and every employee must be accountable for producing results citizens expect. That’s why she is 
changing the culture of state government from one that focuses on programs and processes, to 
one that regularly measures results, improves performance, and identifies and solves problems.” 

Legislative leaders have also proposed a series of performance reporting systems that encourage 
higher education institutions to be more accountable for addressing state priorities. Washington’s 
community and technical colleges are currently required to report their performance on measures 
including job placement and baccalaureate transfer rates. A key Democratic legislator we 



^ http://www.accountabilitv.wa.gov/ 
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interviewed observed, “It’s important to measure accomplishments rather than efforts. How we 
do business in government now is a measure of effort, but the issue is not effort but results!” 



C. Origins of the Student Achievement Initiative 

The current Student Achievement Initiative grew out of the involvement of the SBCTC in the 
Ford Foundation-funded Bridges to Opportunity Initiative, an effort involving Washington and 
five other states, the aim of which is to promote state policies that improve outcomes for 
underprepared adult students. Under the Bridges initiative, the Washington community and 
technical college system became increasingly focused on how to improve colleges’ capacity to 
serve underprepared students. 

In early 2006, as a result of a shift in their leadership, the State Board members launched a 
process of developing a new strategic plan for the system. As part of the Board’s discussions on 
strategic directions, there were substantial debates about the Board’s role generally within the 
system, and specifically about how it might promote improved institutional effectiveness over 
time. Recognizing that budget policy is the State Board’s main source of leverage with the 
colleges, members reasoned that if they wanted to improve colleges’ efficiency and 
effectiveness, this could best be accomplished by changing the incentives that exist in the budget 
process. The link between finance and educational practice often gets lost in the desire of state 
policymakers to respect local control and the role of institutional trustees who have the 
responsibility to ensure that the colleges they represent are meeting the educational needs of their 
communities. But the Board also believed that it had a responsibility to ensure that the colleges 
improve their outcomes over time, demonstrating the efficient use of public resources with more 
attention paid to documenting outcomes and performance improvements. 

For one State Board member, it came down to pushing back against the fixation on counting 
FTEs (full-time equivalents), a measure of the numbers of students enrolling, and instead 
focusing more attention on how many students were finishing and meeting their educational 
goals. 



We would go through FTE allocation formulas, and I thought it was disingenuous. 

Yes, it’s easy to count, easy to tell one story to the legislature for finance, and a 
wonderful marketing device. But it’s disingenuous if we don’t look at quality and 
whether people can meet their goals and objectives at our institutions. Eor me, it’s 
more important to have achieving schools rather than ETE-driven schools. At 
some point you have to have a more articulated game plan with the outcome being 
student achievement. 

Under the leadership of newly hired SBCTC executive director Charlie Earl, subsequent staff 
discussions on issues raised by the Board led to the conclusion on the part of the SBCTC 
leadership that what was needed conceptually was “something grand that would focus on student 
achievement and that would incorporate a measurement system with built-in incentives for 
colleges.” The staff also acknowledged that it is difficult to change institutional behaviors and 
that to influence college outcomes, enough money needed to be on the table to get the attention 
of college leaders. 
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D. Distinctive Features of the Initiative 



Although the Student Achievement Initiative of the Washington community and technical 
college system shares some important characteristics with earlier performance funding efforts 
across the country, it also has some distinctive features. These include the following: 

• Rather than a set of performance measures dictated by legislation or external 
policymakers, the initiative’s measures were recommended by a task force of State Board 
members, college presidents and trustees and faculty union leaders, and reviewed by an 
advisory group broadly representative of the states community and technical colleges and 
functional areas within these colleges. 

• The selection of performance measures by the task force was informed by research on 
student progression patterns conducted by the SBCTC staff and the Community College 
Research Center (CCRC) at Teachers College, Columbia University, which sought to 
identify “leakage points” where students in different types of programs tend to struggle as 
well as “achievement points” associated with an increased probability of success for 
students who attain them.^ 

• The initiative’s achievement point measures reflect the broad continuum of mission areas 
within the community college system, including a strong emphasis on achievement at the 
adult basic skills and college remedial (or “developmental”) levels. 

• By rewarding colleges for increasing the rate of intermediate educational attainments by 
students starting at all program levels, including the most basic ones, the initiative seeks 
to focus colleges’ attention on student progression and to encourage them to adopt 
policies and practices that increase and accelerate the rate at which all students move 
toward program completion, regardless of where they begin. 

• The achievement point data supplied to the colleges is intended to enable them not only 
to measure their performance, but to identify areas for improvement and success and 
evaluate the effectiveness of efforts to improve student progression. Thus, the 
achievement point framework is designed as a tool colleges can use for performance 
improvement, not just for accountability. 

• Whereas other state performance systems report on outcomes annually, the State Board 
staff provides colleges with quarterly data on their achievement point performance, so 
that colleges can monitor their progress throughout the year and get quicker feedback on 
the impact of changes in policy and practice. 



® For details on the background research by CCRC, see D. Timothy Leinbach and Davis Jenkins, Using longitudinal 
data to increase community college student success: A guide to measuring milestone and momentum point 
attainment, CCRC Research Tools No. 2 (New York: Community College Research Center, Teachers College, 
Columbia University), January 2008, which can be downloaded at: 
http://ccrc. tc.columbia.edu/Publication. asp?uid=570 . 
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• The task force chose to focus on measures that were more likely to be within a college’s 
control, as opposed to measures such as job placement rates, which are subject to 
economic changes, and baccalaureate transfer rates, which depend on student choices and 
other factors community and technical colleges cannot influence. 

• Colleges have considerable flexibility in where to focus their student achievement 
improvement efforts. 

• Colleges do not compete with on another for performance funds, but rather are measured 
against their own historical performance. 

• Funds generated through improved performance are intended to be added to colleges’ 
base budget funding so that program innovations can be sustained over time. 

Many states use non-recurring, one-time revenue streams to support their performance funding 
models. The Washington State Board is seeking to convince the legislature to provide recurring 
funding for the initiative on a scale that will provide sufficient incentive for colleges to change. 



The Washington State Board’s efforts to design a set of student- focused performance indicators 
for community colleges has generated substantial interest from other states. The success or 
failure of the Student Achievement Initiative to support improved student outcomes will be 
watched closely. 
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Chapter 3 

Colleges’ Awareness and Understanding of the Initiative 



This chapter describes how college personnel were informed about the initiative, their level of 
understanding of it, and what they saw as its origins and goals. 



A. How College Personnel Learned About the Initiative 

Student Achievement Initiative policy taskforce and advisory committee 

As mentioned, the State Board staff solicited input on the design of the initiative through both a 
policy task force made up of college presidents, trustees, and faculty leaders as well as a 33- 
member advisory committee comprised of representatives from a broad range of functional areas 
at the colleges. Both the task force and the advisory committee members served to channel 
information about the initiative back to campus constituencies. 

Research reports on the use of the initiative database 

In September 2007, SBCTC staff distributed an overview of the initiative with data and analysis 
of the achievement points generated by each college during the 2006-07 baseline year.^ 
Throughout the 2007-08 learning year, SBCTC research staff provided colleges with quarterly 
reports on the number of achievement points attained by each college’s current students 
compared to the baseline year. The SBCTC staff also released three additional research reports 
analyzing achievement point attainment patterns by basic skills students, transfer students, and 
students who received Opportunity Grant financial aid,^° respectively. The reports were intended 
to provide guidance to the colleges on strategies for analyzing their own data and for assessing 
the effects on student achievement of particular programs and policies. Our interviews suggest, 
however, that while some senior administrators and institutional researchers had seen these 
research reports, they were not widely known or read by college personnel more generally. 

ITV meetings 

The State Board staff organized a series of four statewide ITV, or interactive television, sessions 
on the initiative during the 2007-08 learning year. The purpose of these sessions was to inform 
the colleges about the use of the initiative’s database and to facilitate sharing of strategies 
colleges were using, or planning to use, to increase student achievement. The ITV sessions also 
gave colleges a chance to bring together a group of college personnel to begin sharing student 
success strategies. Teams from all 34 colleges participated in these session. 



^ Meeting Washington State ’s Needs for an Educated Citizenry and Vital Economy: An Initiative for Measuring 
Colleges and Awarding Funds for Improving Student Achievement, (SBCTC Research Report No. 07-1). 
^Increasing Student Achievement for Basic Skills Students, (SBCTC Research Report No. 08-1). 

^ Transfer student college-level momentum points (SBCTC Research Report No. 08-03). 

Opportunity Grants: A Progress Report on the Postsecondary Opportunity Program, (SBCTC Report No. 08-4). 
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During the initial ITV sessions in fall 2007, State Board staff gave an overview of the initiative 
and the baseline year data. While some college respondents said these early ITV sessions were 
helpful in understanding the initiative and learning how other colleges were approaching the 
initiative, most of those we interviewed suggested that the ITV format was not conducive to 
achieving the goals of these sessions. Most respondents indicated that they and their colleagues 
left the ITV sessions confused and that participants did not speak up although many clearly did 
not understand the details of the initiative. Instead of the ITV sessions, several college personnel 
said they would like to have had more interactive gatherings that brought college personnel and 
SBCTC staff together to discuss their particular questions and concerns about the initiative. 

The winter and spring ITV sessions were designed to advance a key objective of the initiative: 
the system- wide sharing among colleges of effective strategies to improve student achievement. 
While college personnel were optimistic that the initiative would facilitate the sharing of 
effective strategies, some expressed a lack of confidence in the quality of the evidence to support 
what were referred to in the ITV sessions as “best practices.” For example, one college president 
said she was not impressed with the practices presented during a spring ITV session. She said. 

Some of the “best practices” I didn’t find particularly impressive.... When I hear 
“best practices” I assume there’s some data. Nobody seemed to have that. They 
said “we don’t have data yet, but it feels like we’re getting some good success.” 

And maybe that’s enough, but at what point does it become a “best practice”? 

State Board staff also convened a two-hour ITV session in late spring 2008 that was dedicated to 
technical issues related to the state system data warehouse. While the session was not directly 
related to the Student Achievement Initiative, institutional research personnel described the 
workshop as very helpful in understanding the initiative’s database. 

SBCTC staff presentations at statewide council and committee meetings and at colleges 

State Board staff made presentations at meetings of the system- wide commissions and councils, 
including the instruction commission, the student services commission, and the council for basic 
skills. Staff members also visited several colleges to discuss the initiative and respond to 
questions and concerns from college personnel. These presentations and face-to-face meetings at 
colleges were viewed as extremely helpful in communicating the goals of the initiative and in 
being responsive to questions and concerns. 

College-level working groups 

College personnel engaged in activities related the initiative on their campuses provided a 
potentially valuable source of information about it. Colleges organized fairly consistent, broad- 
based teams of administrators, faculty, and staff to participate in the ITV sessions, and several 
colleges had working groups of college personnel with responsibility for overseeing their 
institution’s work on the initiative. The members of these groups were often a key source of 
information about the initiative for their colleagues and for college personnel in their functional 
areas or departments in particular. At several of the colleges, senior college administrators said 
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they sought to disseminate information about the initiative college-wide through email or in 
discussions about the initiative at all-campus events. 



B. Level of Awareness and Understanding of the Initiative among College Personnel 

College personnel across the state described the purpose of the initiative in language similar to 
that used by the Student Achievement Initiative task force and adopted by the State Board. Those 
we interviewed generally understood that a central purpose of the initiative is to help all students, 
including those unprepared for college-level work, to successfully progress through an 
educational program. There was, however, a wide range of awareness and understanding about 
the specific details of the initiative. College presidents, other high-level administrators, and 
institutional research directors were generally well-informed about the initiative, but few faculty 
and staff had a clear understanding of how it worked. Even among those who were somewhat 
well informed about the initiative, there was a great deal of uncertainty and even confusion about 
the details. 

Presidents and senior administrators 

The presidents had fairly detailed knowledge of the initiative, including an understanding of both 
the achievement point categories and the proposed funding scheme. The promise of additional 
funds through the initiative, combined with the fact that each college’s performance would be 
publicly reported, focused presidents’ attention on the initiative. Most college presidents we 
interviewed had seen their college’s 2006-07 baseline performance data and had an idea of where 
their colleges would focus their resources to earn achievement points. However, most presidents 
we interviewed said they did not see the quarterly updates of their college’s data that were 
provided by the State Board staff throughout the learning year. 

Most vice presidents of student services were also fairly well-informed about the design of the 
initiative. As detailed in the next chapter, student services divisions at a majority of the colleges 
led the response to the initiative, and the initiative’s start-up funds were often used in support of 
student services. Vice presidents of instruction, along with deans and department chairs, were 
generally less knowledgeable than were their colleagues in student services. The majority of 
senior administrators had seen their college’s 2006-07 baseline year data, but few, if any, had 
looked at the college’s quarterly performance data. 

College trustees 

Awareness and understanding of the initiative by college boards of trustees varied. While a few 
of the trustees interviewed had almost no knowledge of the initiative, the majority understood its 
broad goals and design. Several trustees had detailed knowledge of the initiative’s achievement 
point categories and how funding would be linked to performance. One such trustee praised the 
efforts of the State Board staff, beginning with the early stages of the development of the 
initiative, to keep trustees informed: 
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I have to give our State Board [staff] lots of kudos. They have been tremendously 
helpful in disseminating the information to each individual trustee. We get email 
correspondence on an individual basis with short memorandums. If you have a 
question, call this person. If you have a comment, call this person. It’s been that 
way from the beginning. This wasn’t something where a task force met and no 
information was shared until the recommendations were made. 

Institutional research personnel 

The directors of institutional research (IR) and their staff generally had detailed understanding of 
the initiative. In particular, IR directors and staff often had the most knowledge about how points 
are earned across the achievement point categories and about their own college’s performance. 

IR staff members at almost all of the colleges were responsible for understanding how points 
were accrued and for keeping senior administrators informed about their college’s performance. 
At the time of our interviews, however, most IR directors were struggling to understand the 
intricacies of the database being supplied to the colleges by the State Board as part of the 
initiative. One IR director described frustration at having to re-code the data so that the college 
would be properly credited for achievement point gains by its students, saying. 

For some people it’s as clear as mud how they [State Board staff] are counting 
things, and so we are all struggling with that. This quarter we spent quite a bit of 
time re-coding students who were not properly coded... so we could accurately 
count them. We will continue to do that. 

Institutional researchers from across the state have established a state- wide working group and 
electronic mailing list to facilitate discussion and sharing about the initiative’s database. 

Faculty and student support services staff 

Beyond a small group of instructors and student support staff directly involved in related 
activities at their colleges, there was not broad awareness of the initiative among other faculty 
and student support services staff at most of the colleges in our study. Student services staff 
generally had a more detailed understanding of the initiative than did most faculty. The majority 
of faculty members we interviewed had only a limited understanding of the initiative; only a few 
had any detailed knowledge or had seen their college’s 2006-07 baseline year performance data. 
The description of faculty awareness by a vice president of instruction was typical: 

With our faculty we’ve told them that this initiative is happening. . . . Faculty know 
that something is happening, but that is the extent of it.... The faculty have had it 
explained to them, but if you talked to them, they couldn’t explain it back. 

Many respondents — both faculty and student support services staff — could not, or at least did 
not, distinguish between ongoing efforts to promote student retention and success at their college 
and activities planned in response to the Student Achievement Initiative. Some said internal. 
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ongoing student success initiatives such as Achieving the Dream and Title III activities were 
much more visible. 

Interviews with college personnel suggested several possible reasons for the general lack of 
knowledge about the initiative among faculty and staff. First, at a few colleges, senior 
administrators had made concerted efforts to inform faculty and staff about the initiative, 
including by email, through department and faculty meetings, and at all-college events. Yet, 
administrators we interviewed reported that college personnel, and faculty in particular, 
generally had not paid attention to the initiative because it had not yet directly affected their 
work. Several administrators stressed that the colleges were still in the “learning year” of the 
initiative. They expected awareness and understanding of the initiative to increase over time. 

Second, the common lack of strong connections between basic skills and college-level faculty, as 
well as between student services and academic programs, seemed to reduce communication 
about the initiative at many colleges. As discussed in the next chapter, most of the colleges 
focused their initial efforts in student services and basic skills during the learning year. As a 
result, personnel with the greatest understanding of the initiative were primarily working in those 
areas. Because basic skills has historically operated at the margins of many colleges, awareness 
and interest in the initiative among basic skills instructors had not spread to college-level faculty 
at at least some colleges in our study. 

Third, college administrators at a few colleges purposefully limited communication about the 
initiative to faculty and staff. Some college administrators said they did not want to distract 
attention and energy from existing student success efforts by touting a new external directive 
from the State Board. At some colleges, presidents and senior administrators were taking a “wait 
and see” approach to the initiative. The uncertainty about the availability of future reward money 
to support the initiative led some college personnel to conclude that the initiative may be short- 
lived. These college administrators expressed a reluctance to involve faculty in a temporary 
initiative that could well be resisted by faculty. 

At other colleges, senior administrators reported that they themselves did not fully understand 
the specifics of the initiative, particularly about how the achievement points were derived. Many 
college administrators also questioned the accuracy of their achievement point counts. They were 
reluctant therefore to try to inform faculty and staff about the initiative. At one of the smaller 
colleges, the vice president of instruction provided only limited information about the initiative 
because of perceived underreporting of the college’s baseline achievement points: 

I have brought it [Student Achievement Initiative] up with faculty about where 
it’s going..., but I haven’t shared it all with them. When you give data to faculty 
and you know it is wrong, it creates more challenges than if we wait until it gets 
cleaned up and then you move forward. 

The following table summarizes our assessment of the overall level of awareness and 
understanding of the Student Achievement Initiative at the 17 colleges where we conducted 
interviews. At a majority of the colleges, awareness of the initiative was limited. Several colleges 
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were taking steps to inform faculty and staff about the initiative and garner support for it. At no 
college was there widespread understanding about the details of the initiative. 



Awareness and Understanding of the Initiative 


Number of Colleges 


Minimal - Little or no awareness 


0 


Limited - Some administrators aware and knowledgeable; 
limited faculty and staff awareness 


10 


Moderate - Administrators generally aware and 
knowledgeable; concerted efforts to inform faculty /staff about 
initiative details 


7 


Extensive - Administrators, faculty, and staff across tbe college 
are familiar with initiative and at least some specifics 


0 



Need for continued communication about the initiative 

Several respondents praised the substantial efforts by SBCTC staff to inform college personnel 
about the initiative. State Board staff members were seen as very responsive to questions from 
college personnel. Yet, it is clear that many rank-and-file members within colleges have not 
heard about the initiative. Among those who were aware of the initiative, there was a great deal 
of uncertainty and even confusion about the details, particularly around how future funds would 
be allocated to colleges. 

At the majority of colleges, interviewees indicated that they welcomed or needed additional 
communication and guidance from State Board staff. College trustees and presidents said that 
they wanted additional information about the initiative, both for their own understanding and for 
informing a broader audience in their colleges and districts. College personnel said that the 
printed communication and reports from SBCTC staff were generally too long and technical and 
therefore of limited use to most faculty and staff in understanding the initiative. College leaders 
described a pressing need for a summary document that succinctly communicates in “talking 
points” language how achievement points are determined and how incentive funding will be 
allocated.*^ One long-serving president who was generally supportive of the initiative described 
the issue this way: 

I like it [the Student Achievement Initiative] because it looks at the full scope of 
what we do. But it’s an incredibly complicated scheme. For someone like me who 
needs to paraphrase and generalize, it’s been very difficult to come up with a 
succinct, executive summary of what it’s about. 

An institutional researcher at one college who is well-regarded across the state emphasized the 
need for better communication around the achievement point data in particular. 

I think if they had spent more time with institutional researchers — getting them 
in tune with the data, fa mi liar with the data system, how it was set up, and really 
giving us the tools — I think we could have done more than we have in a year. A 
year is still not enough, especially if you want to get momentum at a community 



** There is a two-page “FAQ” document on the SBCTC website, but none of those we interviewed mentioned it. 
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college [and] get people involved and understanding. [It is difficult] trying to 
explain what the points mean.... [Faculty and staff] don’t care about the points, 
but they need to care about the points. 

The lack of awareness and clarity about the initiative should not be surprising. The initiative was 
developed and is being implemented on a relatively short timeline, and the colleges were still in 
the first “learning year” of the initiative during our interviews. Awareness and understanding of 
the initiative — in terms of both the achievement point performance indicators and the funding 
process — will likely increase with time. Still, college personnel at several colleges suggested 
that clear and transparent communication about the initiative would allay some of the uncertainty 
and anxiety surrounding the effort and help build support for it. 



C. Understanding of the Origins and Goals of the Initiative 
State Board seen as the primary driver 

Interviewees at all 17 of the colleges where we conducted interviews viewed the State Board as 
the primary driver of the initiative. College personnel described the initiative as part of a larger 
trend in which the State Board has assumed a more active role in system governance. College 
presidents and senior administrators said that there has recently been a shift in the State Board’s 
relationship with the colleges, with a gradual but noticeable decrease of college autonomy. Many 
college leaders and senior administrators seemed to view the Student Achievement Initiative in 
this context — as one of a number of recent directives coming from the State Board. The director 
of basic skills at one college described the tension between the colleges and the State Board this 
way: 



In the past the State Board has not played as strong a central role as it is right now 
in something like this. In the past the State Board might have said something, the 
presidents may have taken it as a suggestion, spent a lot of time thinking it over, 
and then come back with a counter proposal. In this situation, this was set up by 
the State Board, and I think the State Board’s position is, “No, you don’t 
understand. We are telling you this is something you all are going to do.” And so 
the colleges have been slower to respond.” 

A few college respondents — particularly trustees and college presidents — mentioned that 
turnover on the Board in the last couple years brought new members with an interest in 
measuring educational outcomes. The president of one institution described the State Board’s 
emphasis on using incentives to drive institutional behavior, saying, “I think it [the Student 
Achievement Initiative] was coming directly from the State Board itself. I think they are a 
business-oriented board, an achievement-oriented board, a board that appreciates the use of 
incentives.” 

Some college respondents said they believe that the initiative represents a strategy by the State 
Board for achieving the 10-year strategic goals it recently established for the system, particularly 
the two related to strengthening state and local economies by meeting the demands for a well 
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educated and trained workforce, and increasing educational attainment among all residents 
across the state. 

Response to growing pressure from policymakers for increased accountability 

A wide range of college personnel — from trustees to counselors — said that calls for 
educational accountability have been growing in recent years from several sources — including 
state and federal governments and accreditation agencies — and that accountability will likely be 
a feature of the higher education landscape for the foreseeable future. Some college personnel 
indicated that accountability is a priority across all areas of state government, and a few 
specifically referenced Governor Gregoire’s Government Management Accountability and 
Performance (GMAP) program mentioned earlier. An instructional dean at one of the colleges 
described the legislature’s interest in measuring the quality of state services this way: 

Our legislature has been pretty keen on measurable results in all kinds of 
categories, and higher education is not an exception to that. We have a lot of 
people around the legislative table [who] are saying, “show us what you are 
doing for the resources we are investing in you.” This is exactly the kind of thing 
we will probably see more of, not less. 

Interviewees at 12 colleges suggested that the Student Achievement Initiative was created in 
response to accountability pressures from the state legislature. Several college leaders indicated 
that the state legislature is looking for reliable and quantifiable data on college performance and 
described the Student Achievement Initiative as an attempt by the State Board to develop a 
performance accountability system for the system before the legislature imposed its own. One 
president called the initiative a “pre-emptive strike” by the State Board: 

We know that this is probably a good move from the State Board point of view, to 
define us before someone from the outside comes and defines us. I think you are 
in a better position if you can go to the legislature and say, “you don’t need to 
worry about it [accountability]. We already have it in place and here it is. 

College trustees, presidents, and senior administrators often held different views than faculty and 
staff on the meaning of accountability vis-a-vis the initiative. College leaders and senior 
administrators were more likely to view the initiative as a means to secure additional future 
funding by providing quantifiable evidence of improvements in student achievement. Faculty 
and staff were less likely to understand the initiative in terms of an incentive for performance 
improvement, and more likely to frame the purpose of the initiative in terms of providing an 
external assessment of their work. As a result, faculty and staff generally had a more negative 
view of accountability pressures than did college trustees and presidents. 

Influence of research by State Board staff on the initiative’s design 

Interviewees at all but one of the 17 of the colleges we studied mentioned the influence on the 
initiative’s design of the State Board staff’s research on student progression and outcomes. 
According to a vice president of student services. 
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The State Board’s involvement in the [Ford Foundation-funded] Bridges project 
— that’s where it first came from. The State Board did some research within the 
Bridges project and they started talking about these [achievement] points where 
students make progress. When the Bridges project ended, then this new concept of 
[achievement] points began. 

There was widespread respect for the State Board’s research on student progression and 
achievement, even among college respondents with reservations about the initiative overall. For 
example, one faculty member who opposed tying funding to institutional performance 
nonetheless commented on the quality of the research used to design the achievement point 
framework, saying simply, “You can’t argue with the research.” 

Input by college personnel into the design of the initiative 

While those we interviewed at colleges generally agreed that the impetus for the initiative came 
from the State Board (and some viewed the initiative in terms of a “top-down” mandate), several 
highlighted the input that college representatives had in the design of the initiative. Specifically, 
respondents mentioned the representation of college presidents on the task force that 
recommended the design to the Board as well as of colleagues who served on the system-wide 
advisory committee. At almost half the 17 colleges in our study, presidents spoke positively 
about the role that they or their colleagues had in shaping the initiative and the design of the 
incentive funding mechanism in particular. 

A few college administrators said the efforts of the State Board and State Board staff to solicit 
feedback from the colleges as they developed the Student Achievement Initiative were a 
welcome contrast to the state’s previous performance funding scheme enacted in the late 1990s. 
According to one administrator, unlike the earlier policy, which was conceived by one powerful 
state legislator with little input from the Board or the colleges, this one was designed with broad 
input from the colleges. 
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Chapter 4 

Colleges’ Initial Approach to the Initiative 



During the 2007-08 learning year, colleges were expected to become familiar with their 
achievement point data and begin to develop and share strategies to improve student 
achievement. This chapter explores colleges’ initial approach to the Student Achievement 
Initiative and describes the progress they made and the challenges they encountered during the 
learning year. 



A. Organization of Implementation Efforts 

Seven of the 17 colleges included in our study established a new committee or working 
group to plan and coordinate activities related to the initiative. The other ten colleges 
delegated oversight of the initiative either to standing committees with responsibility for 
student success or to semi-structured groups of individuals involved with strategic 
planning, enrollment management, or student success at their institutions. These 
committees and working groups included senior administrators from instruction and 
student services, as well as student services staff, institutional researchers, and faculty 
members, most often from basic skills or developmental education. 

Representatives from these new or existing groups participated in the ITV sessions and, 
at four colleges, met frequently throughout the year to discuss the initiative. At most 
colleges, however, the level of engagement with the initiative by these groups was 
modest. For example the leader of a working group at one college said that most of her 
group had participated in the ITV sessions, but they had met just one other time during 
the year to review the college’s baseline data. 

Focus on student services 

Nine of the 17 colleges in our study gave primary responsibility for responding to the initiative to 
their student services division. Colleges tended to house their initiative activities within student 
services for at least two reasons. First, college personnel at several colleges said that the 
initiative’s $51,000 start-up funds were targeted to serve 50 “TRIO-like” students, a reference to 
the federal outreach and support programs that serve low-income students, first generation 
students, and those with disabilities. (Several college personnel said that, at the beginning of the 
learning year, they did not understand that these funds, originally called “persistence funds” by 
the State Board, were even part of the Student Achievement Initiative.) The language of the 
legislation allocating the funds for the initiative to the SBCTC included the term “TRIO-like,” 
which was then conveyed to the colleges by SBCTC staff. Since oversight of TRIO programs is 
generally the responsibility of student services divisions, some colleges decided to assign the 
Student Achievement Initiative to those divisions as well. 

Second, at most of the colleges where we conducted interviews, student services divisions have 
primary responsibility for efforts to improve student retention, so it made sense for them to take 
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the lead on this new effort concerned with student progression. Student services personnel were 
enthusiastic about what they saw as an increased emphasis on student persistence by their 
colleges. Most indicated that their existing student retention efforts would help to improve their 
college’s performance on the achievement point measures of the Student Achievement Initiative 



B. Use of the Student Achievement Initiative Database 

While senior administrators and institutional research staff at most colleges recognized the 
potential of the initiative database to track student progress over time and monitor the impact of 
efforts to improve student progression and success, few colleges had actually begun to use the 
database for these purposes during the learning year. Interviews with IR directors and other 
college personnel revealed that most colleges had performed only a cursory analysis of their 
performance using the achievement points. Senior college administrators in student services and 
instruction had generally seen their college’s 2006-07 baseline year data, but only a few had 
looked at their performance data beyond the 2006-07 year. Few faculty or staff had seen any data 
on their college’s performance. 

The following table summarizes our assessment of the extent to which the initiative database is 
being used across the 17 colleges where we conducted interviews. 



Use of the Database 


Number of Colleges 


Minimal - Review by IR staff only 


1 


Limited - IR staff and some senior administrators have seen 
baseline data; IR staff have identified coding and other issues; 
some effort has been made to identify areas of success and 
areas for improvement 


12 


Moderate - IR staff has good grasp of college’s data; college 
has begun to use data to identify areas of success and areas for 
improvement 


4 


Extensive - Administrators and IR staff have analyzed the 
college’s data; faculty and staff have been involved in 
examining and using information from tbe database to design 
improvement strategies 


0 



In attempting to use the achievement point data, colleges encountered a number of problems, 
which are described below. 

Uncertainty about the definitions, accuracy, and integrity of the data 

Colleges reported that they struggled throughout the learning year to understand the Student 
Achievement Initiative database. Several colleges had difficulty figuring out how the State Board 
staff calculated the achievement point totals for their colleges in the 2006-07 baseline year. IR 
personnel and administrators often questioned whether the data provided an accurate measure of 
their college’s performance. Several colleges argued that their achievement point totals were 
underreported due to the miscoding of students and courses. IR personnel said that the SBCTC 
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staff was revising the 2006-07 baseline year data based on consultation with the colleges, and 
many colleges were waiting until the revised 2006-07 baseline year data were finalized before 
conducting more extensive analysis of the data or sharing information about their college’s 
performance with a broader college audience. While questions were occasionally raised about 
the accuracy of the database, college respondents typically expressed great confidence in the 
competence and objectivity of the State Board staff responsible for designing and managing the 
achievement point database. 

Inadequate IR capacity 

How much progress colleges made with analyzing their achievement point data largely depended 
on their institutional research (IR) capacity. At several colleges, institutional research has not 
been a priority, so in those cases there was a shortage of staff with the time or skills to rigorously 
analyze the achievement point data. The vice president of instruction at a college with a small IR 
staff said, 

[The State Board research staff] needs to give us the packaged report that they 
have so that we can duplicate it on our campus. Every college is coming up with 
different queries to try to get out this information themselves. My colleagues at 
other colleges are also having problems. We have a common dataset, but we 
aren’t all pulling data out the same way. 

Even at colleges with larger IR departments, college personnel suggested that the achievement 
point database does not provide enough information to pinpoint areas of weakness, let alone 
design improvement strategies or track the progress of ongoing student retention efforts. As a 
result, colleges have to use their own data to do such analyses, and there is wide variation in the 
capacity of colleges to do so. 

Additional guidance needed 

Several IR directors said that, while they appreciated the SBCTC staff’s help in understanding 
how the college’s achievement point data were derived, they needed additional training and 
assistance both to be confident in the integrity of the data and to understand their college’s 
performance. 

As mentioned, the statewide council of institutional researchers convened a working group to 
discuss data issues related to the initiative. Also, the spring 2008 ITV session for IR personnel 
was described as a big help in understanding the data. IR personnel recommended additional 
training sessions led by SBCTC staff for IR staff and other potential “power users” as well as 
additional sample queries generated by SBCTC staff that colleges could use to conduct 
preliminary analysis of the data. 

The need for greater clarity around the Student Achievement Initiative database will have to be 
addressed before the data will be used more extensively. As indicated earlier, administrators at a 
few colleges said that they were reluctant to disseminate analyses to a wider college audience 
until they had a better grasp on the data. 
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C. Strategies for Increasing Student Achievement 

A central aim of the Student Achievement Initiative is to motivate colleges to identify and 
implement strategies for increasing the rate of student achievement. The following table 
summarizes our assessment of progress across the 17 colleges in developing strategies for 
improving student outcomes using the initiative’s database. 



Development of Success Strategies Using Initiative Data 


Number of Colleges 


Minimal - Little or none 


3 


Limited - College has begun to think about how it might use 
initiative database to develop strategies for increasing 
achievement, but currently relying on existing strategies 


10 


Moderate - College has begun preliminary analysis of the 
initiative database to evaluate existing strategies and develop 
new ones; piloting of new strategies likely in 2008-09 


4 


Extensive - College makes extensive use of initiative database 
to develop and evaluate student success strategies 


0 



As is evident from the table, during the learning year, most colleges were not yet at the stage of 
implementing new programs and services to increase the rate of achievement point attainment by 
their students; instead they were relying on existing student retention and efforts to do so. 

At the same time, all 17 of the colleges in our sample already had student success efforts 
underway. Some had received or were applying for federal Title III grants that focused on 
student retention and success. Some were using institutional funds from the state’s Opportunity 
Grants financial aid program to provide wrap-around support services to better retain 
economically disadvantaged students who received these financial aid grants. A few colleges 
were participating in Achieving the Dream, a national initiative focused on improving retention 
and completion of traditionally underrepresented students. 

Use of initiative “start-up” funds 

One college reported that it put the $51,000 in start-up funds from the initiative into its general 
operating budget. Another spent the funds for institutional research support. However, the other 
15 used the start-up funding mainly to support staff salaries or provide operational funding for 
existing student success efforts. A list of the ways the colleges we studied spent their start-up 
funds is presented in the box below. 
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Uses of Student Achievement Initiative Start-Up Funds 
by Colleges During the Learning Year 

Instruction 

• Release time for faculty to develop developmental math success strategies (1 college) 

• Support for additional developmental math course offerings (1 college) 

• Enhanced learning and support with counselors working closely with instructors in the 
classroom — based on a model for teaching students with disahilities (1 college) 

• Contribution to innovation fund providing “mini-grants” to faculty with proposals for 
ways to improve student success (1 college) 

Tutoring 

• Support for tutoring center (1 college) 

• Tutoring support (1 college) 

• Support for on-line tutoring service (1 college) 

Assessment 

• Staff support to implement improved documentation of CASAS testing results for basic 
skills students (3 colleges) 

Advising 

• Salary support for student achievement coordinator to facilitate student transitions from 
pre-college to college-level coursework (2 colleges) 

• Supplemental case management for at-risk students (1 college) 

• Peer mentoring support for underrepresented students (1 college) 

• Salary support for persistence advisor/specialist (1 college) 

• Support for MAPS: Mentoring and Advising for Persistence and Success (1 college) 

• Support for TRIO Upward Bound program (1 college) 

• Salary support for financial aid counselor (1 college) 

• Support for new student success center providing “one-stop” advising, admissions, 
registration, and financial aid services (1 college) 

• Redesign of advising program for first-year students (1 college) 

Career Advising 

• Support for career advising during orientation (1 college) 

• Support for career exploration program to assist former high-school drop-outs with career 
advising ( 1 college) 

• Salary support for career advisor for first-year students (1 college) 

Other 

• Supplement math textbook scholarship fund for low-income students (1 college) 



CCRC Formative Evaluation of the Student Achievement Initiative “Learning Year 



30 






Focus on basic skills and developmental education 

Some colleges focused their initial work on the initiative on basic skills and developmental 
education. There were several reasons for this. 

To begin with, SBCTC staff responsible for basic skills made a concerted effort to inform the 
Council for Basic Skills of the initiative, which helped raise awareness among basic skills 
programs directors. The director of basic skills at one college where basic skills was a focus of 
the initiative described how he had first heard about the initiative from a State Board staff 
presentation at a Council for Basic Skills meeting the previous year: “Initially, I was more aware 
of the [Student Achievement Initiative] than most of the other players at the college. I was in a 
good position to put forward the suggestion for basic skills [to be a focus of the college’s work 
on the initiative] and got approval for it.” 

There was a widespread perception that colleges could earn more points in the basic skills 
achievement category than in those related to passing college-level math, completing college 
credits, or earning credentials. A dean at the same college as the director of basic skills cited just 
above explained why it made sense to emphasize basic .skills: 

The Student Achievement Initiative gives more points proportionally for students 
who are making small steps early on [at the basic skills end] than [for students] at 
the upper end of the progression. . . . This puts more explicit focus on a category of 
students who have not been a priority and have not been tracked as well as they 
should be. 

At least 9 of the 17 colleges had begun efforts to improve outcomes in basic skills and 
developmental education before the advent of the Student Achievement Initiative. Interviewees 
said that the initiative provided additional impetus to strengthen those efforts. Several college 
administrators said that there has been a growing recognition in recent years that too few 
students progress through the various levels of basic skills or of developmental education and 
advance to college-level coursework. A vice president of instruction described the incentive that 
the initiative provided her college to better understand and address the barriers to success 
confronting basic skills students: 

It is interesting to track our basic skills students as to how they are progressing — 
not only through basic skills but on to pre-college and college-level work. We 
know our rates are low, and why is that? This [Student Achievement Initiative] 
does give us an incentive to get a better handle on that. 

Basic skills programs were often perceived to be more receptive to the initiative because they are 
accustomed to being held accountable for student performance. Basic skills directors and 
instructors at several colleges suggested that existing state and federal performance requirements 
had given them experience tracking the progress and outcomes of their students. One director of 
basic skills at a college that had centered most of its Student Achievement Initiative activities in 
basic skills said that he and his colleagues were well-positioned to respond to the initiative. 
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“There is a lot of buy-in from the basic skills area of colleges because we have 
been exposed to performance measures, collecting data, and having outcomes 
measured.” 

Planned efforts 

At least 9 of the 17 colleges were planning to implement new measures in the coming year to 
increase achievement point attainment by their students. Among the specific approaches 
proposed were: 

• Conducting “degree audits” or transcript reviews to identify students who achieved the 
required credits for a certificate or degree but never received one; 

• Improving documentation of CASAS testing results for basic skills students; 

• Eliminating certificate or diploma fees; creating academic certificate programs; 

• Increasing credit requirements for certain shorter occupational certificate programs so 
they will qualify for achievement points. 

Note that most of these efforts are likely to result in one-time gains in achievement points for the 
colleges in question. 

College administrators had also begun aligning the initiative’s performance measures with their 
strategic planning and re-accreditation activities. Six colleges had begun early efforts to 
incorporate the initiative into their internal strategic planning processes. College personnel at 
eight colleges discussed how the initiative’s focus on documented student progression was very 
consistent with their institution’s efforts and strategies for re- accreditation. 



D. Factors Affecting Colleges’ Receptiveness to the Initiative 

Not surprisingly, the interviews with college personnel revealed variation across the colleges in 
both the level of receptiveness to the Student Achievement Initiative and the extent to which 
colleges had begun the process of planning and implementing responses to it. Several factors 
seemed to affect a college’s response to the introduction of the Student Achievement Initiative. 

Enrollment trends 

The CCRC research team heard widespread concern about stagnating or declining student 
enrollment numbers across the state. Leaders at several colleges said that the Student 
Achievement Initiative has the potential to help ameliorate the effects of lower student 
enrollments because colleges could accrue additional funds not directly attached to FTEs, and 
successful student retention efforts implemented under the initiative would lead to increased 
student enrollment. 
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Leadership support for the initiative 



Not surprisingly, colleges where the leadership was noticeably supportive of the initiative seem 
to have made the most progress in building support for it among college stakeholders. 

Perceived/expected faculty resistance 

Administrators at several colleges suggested they would welcome assistance from SBCTC staff 
on how to effectively communicate the initiative to a broader college audience. Some college 
administrators suggested they had purposefully not tried to build awareness because of perceived 
or expected pushback against the initiative from the faculty. Leaders at some colleges wanted to 
be sure the initiative was seen by faculty and staff as fitting into the existing goals and processes 
by which the college measures performance rather than as an external state mandate. 
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Chapter 5 

Colleges’ Views on the Initiative’s Performance Measurement Model 



To identify appropriate measures of student achievement for Washington’s community and 
technical colleges, the State Board staff reviewed research on community college student 
progression and outcomes. In partnership with the Community College Research Center, the 
State Board staff also analyzed their own system- wide data on student “momentum” to identify 
intermediate education attainments associated with a higher probability of success for students 
starting at different program levels. This research was reviewed by the advisory committee, 
which then presented its findings to the Student Achievement Initiative policy task force. The 
achievement point performance measures recommended by the task force and adopted by the 
State Board reflect four principles: 

• Performance measures recognize students in all mission areas and reflect the needs of the 
diverse communities served by colleges. 

• Performance measures measure incremental gains in students’ educational progress 
irrespective of mission area. 

• Measures represent simple, understandable, reliable and valid attainments in students’ 
educational progress. 

• Measures focus on student achievement improvements that can be influenced by colleges. 

This chapter examines the views of college personnel on the initiative’s performance 
measurement model. It describes the aspects that those we interviewed said they like about the 
model as well as their concerns about it. 



A. Support for the Initiative’s Performance Measurement Model 

Most of those we interviewed viewed the achievement point framework favorably. The 
following are some of specific aspects that were frequently mentioned in a positive light. 

Increased focus on student success 

Several interviewees reported that the initiative has helped to draw attention to student success 
efforts at their college. College personnel at 12 colleges indicated that the initiative has led to 
increased dialogue about barriers students face and about strategies for helping students 
overcome those barriers. 

Means of measuring program effectiveness and student success 

Many of those we interviewed applauded the fact that the Student Achievement Initiative 
provides both an incentive and a means for colleges to measure the effectiveness of efforts to 
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promote student retention and success. Some basic skills and career-technical administrators and 
faculty reported that their departments had a lot of experience tracking their students, but most 
college personnel we interviewed said that they had not done an adequate job in the past of 
monitoring student outcomes and evaluating the effectiveness of their programs and services. 

While many interviewees questioned the accuracy of their baseline year achievement point 
totals, few voiced concern about the validity of the performance measure categories. As 
mentioned, college personnel generally expressed widespread respect for the quality of the 
“tipping point” study and other research used by the State Board in developing the achievement 
point performance measurement framework. 

As discussed in the previous chapter, senior administrators at some colleges discussed plans to 
use the achievement points to set goals as part of their strategic planning efforts. Other 
interviewees described how the achievement measures would benefit their college’s re- 
accreditation efforts, specifically by providing a means of documenting and continually 
improving student progression. 

Measures recognize students in all mission areas and focus on student achievement within 
colleges’ control 

A particular strength of the model in the eyes of many interviewees is that it measures students’ 
progression from basic skills through completion of degrees. Several pointed out the contrast 
with previous performance measurement systems, which did not give any credit to colleges for 
helping students attain intermediate achievements and which included outcome measures, such 
as baccalaureate transfer rates and job placement rates, that are to some extent beyond the control 
of colleges. A director of institutional effectiveness put it this way: 

We’ve had other accountability measures in the past. I view the Student 
Achievement Initiative as doing a better job of measuring what the colleges are 
accomplishing. The last set we had for a decade only measures outcomes — 
whether they transferred or got jobs. If a student attended for two quarters and 
left, the college couldn’t count them as an outcome. In the Student Achievement 
Initiative, if a student comes and moves forward in their college career, we get 
some points for what they did here. 

Increased attention to basic skills and developmental education 

The initiative has brought increased attention to basic skills and developmental education, 
particularly developmental math. Colleges across the state viewed basic skills as an area where 
they could make substantial achievement point gains. While recent system- wide initiatives such 
as I-BEST had already increased attention to the need to increase the rate at which students 
transition from basic skills to college, the initiative was seen as further encouraging that trend. 
Similarly, college personnel discussed a statewide recognition that progression through 
developmental and into college-level math remains a significant challenge for students, and that 
the Student Achievement Initiative provides an additional incentive for colleges to help students 
take and pass math courses. 
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Increased investment in student services 



Several student services administrators said that the Student Achievement Initiative has 
highlighted the importance of their work in the eyes of their college’s leadership. Specifically, 
the initiative has helped to draw attention to the importance of “wrap-around” services to 
increase student retention and completion. A few student services personnel discussed how the 
Student Achievement Initiative, as a high-profile initiative of the State Board, had already led 
their college administration to steer internal college funds (in addition to Student Achievement 
Initiative start-up funds) to their divisions. 



B. Concerns about the Model 

Despite considerable support for the initiative’s performance measurement model, our interviews 
with college personnel did reveal concerns about it. Most of these centered on whether the 
performance measures would change colleges’ priorities and if some colleges would be 
disadvantaged compared to other colleges. 

Could alter college priorities with unforeseen consequences 

At nine colleges, some of those we interviewed suggested that the performance measures may 
not be entirely consistent with their college’s mission or best reflect the needs of their particular 
students. These respondents mostly expressed a general concern that a State Board initiative 
would not reflect their college’s priorities. A few college personnel were more specific. For 
example, while most of those we interviewed welcomed the focus on basic skills and 
developmental math, a few respondents felt this might lead to an inappropriate shift of 
institutional focus and resources away from college-level coursework to remedial instruction. A 
faculty member who teaches college-level math expressed the concern this way: 

I think there is concern among faculty teaching pre-calculus and calculus or other 
college-level courses. I mean how much do we fund developmental education at 
the community college, and when you start transferring those resources into 
developmental education? Does that mean we will be losing our calculus classes? 

Do you cut back on literature courses and offer more sections of developmental 
education? I mean, are we a college at that point? 

Similarly, some college personnel raised concerns about what might “get left off the table” as 
their colleges invest resources to gain achievement points. While most of those we interviewed 
supported the focus of progression within the college, some, particularly those at technical 
colleges and transfer-oriented community colleges, felt that the initiative’s achievement 
framework is too limited and should include job placement and transfer measures. According to 
a dean at one of these colleges. 

There is no credit for students who transfer, which is really one of our big [areas 
of focus]. The workforce folks, we don’t just want them to have degrees or 
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certificates, we want them to get a job. So, some of those end points are missing 
for me. 

Certain kinds of colleges could be unfairly disadvantaged 

Again, while most indicated support for the overall design of the achievement point framework, 
interviewees frequently expressed anxiety that their college might be disadvantaged by it. These 
respondents suggested that institutional performance across the achievement point categories 
could be influenced by student demographics and other institutional characteristics. Colleges 
feared to be at a possible disadvantage included: 

• Colleges serving more at-risk students. Some of those we interviewed pointed out that 
since underprepared students often require costly wrap-around services to help them 
succeed, colleges serving higher proportions of these students might find it more difficult 
to earn points than those with more students who are prepared for college. 

• Institutions with an academic transfer mission/focus. College personnel at 6 of the 17 
colleges were concerned that the initiative might disadvantage institutions with a focus on 
academic transfer, since the highest potential for earning points seems to be in the basic 
skills and developmental areas. 

• Institutions without capacity for significant enrollment growth. Although the initiative is 
intended to improve student retention and completion, interviewees at several colleges 
pointed out that colleges could earn points by increasing enrollment rather than 
improving the rate at which students progress and succeed. This would put institutions 
whose local student markets do not offer potential for enrollment growth at a 
disadvantage over those in areas with growing demand. 

• High performing colleges. At 7 colleges, interviewees said they were concerned that high 
performing institutions have less room to improve, resulting in a reduced potential for 
earning achievement points compared to lower performing colleges. 

It should be noted that these anxieties were expressed before colleges had a chance to see their 
performance during the learning year. Most colleges were taking a wait-and-see attitude until the 
first year performance numbers became available. 
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Chapter 6 

Colleges’ Views on the Initiative’s Incentive Funding Model 



The Student Aehievement Initiative poliey adopted by the State Board outlined five Principles 
for Incentive Funding: 

• Colleges are rewarded for improvements in student aehievement. 

• Funding is struetured so that eolleges eompete against themselves for eontinuous 
improvement rather than eompeting with eaeh other. 

• Funding is stable and predietable, and eumulative over time. 

• Incentive funding rewards student success and becomes a resource for adopting and 
expanding practices leading to further success. 

• New funds provide the greatest incentive. 

This chapter presents the views of college personnel on the incentive funding component of the 
initiative. 



A. Source of Additional Funding 

College personnel frequently said their institutions were under-funded by the state legislature. A 
few presidents and senior administrators said the initiative, by providing quantifiable evidence of 
student achievement, could serve as a useful means of securing additional money from the 
legislature. One president was hopeful that the initiative would provide a source of funding for 
the college that is not based on growth in enrollment. 

It’s a way to work with the legislature to get funding for something other than 
growth FTEs. Just talking about quality doesn’t do it — we’ve tried that in the 
past — but we didn’t quantify it. Go in there with accountability and say: “Here is 
what you are going to buy with this money in terms of student achievement. Give 
us this amount of money and we will increase student success and quantify it 
using the achievement points. 

The fact that the financial rewards would be permanent additions to their colleges' base budgets 
was particularly appealing to college leaders that supported the initiative's funding mechanism. 



B. Concerns about Incentive Funding 

While the current amount of funding attached to the Student Achievement Initiative is small in 
relation to the overall system- wide budget, the initiative nonetheless represents a potentially 
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dramatic shift in the way community and technical colleges in Washington are funded. College 
personnel across the state are accustomed to a funding model based largely on student 
enrollments for base-budget funding and categorical grant programs for supplemental funding. 
While the colleges already have numerous student retention efforts in place, they currently have 
little financial incentive to improve student outcomes. Perhaps it is not surprising then that our 
interviews with college personnel across the state revealed substantial uncertainty and anxiety 
around the new performance funding model. These concerns are summarized below. 

Could lead to lowering of academic standards 

At several colleges, faculty and academic administrators we interviewed suggested that, with 
funding at stake, there was the potential for inappropriate pressure on instructors to lower their 
academic standards so that more students pass so their colleges can gain points. For example, the 
vice president of instruction at one college cautioned that “the downside is if you don’t do this 
systematically and methodically it’s going to be perceived that we are pushing faculty to change 
their standards ... so students will get the green-light to pass on.” Several interviewees argued 
that the funding for achievement point gains might have a similar negative effect as that of the 
Washington Assessment of Student Learning (WASL), the state’s test for K-12 education, which 
has encouraged teachers to “teach to the test.” 

Might encourage unfair comparisons and discourage sharing of effective practices 

One of the Student Achievement Initiative goals is to increase the sharing of best practices for 
student success across the state. Several college personnel indicated that the Washington State 
community and technical college system has a strong culture of sharing ideas, and predicted that 
the initiative will likely serve to encourage these efforts. Several college respondents said that 
they supported the initiative because they were persuaded by the State Board that colleges are 
competing only against their own past performance, not against other colleges. According to one 
college dean. 



The plan was designed to compete with yourself. That’s the message we have 
received the from State Board, at least coming through the data folks. This is not 
about comparing ourselves to other colleges. We are competing against ourselves 
and how we do over time. 

However, respondents at at least 8 colleges expressed concern that attaching funding to the 
initiative will encourage unhealthy competition among the colleges. Some interviewees argued 
that because the data will be publicly reported, colleges will be concerned about how their 
institution ranks compared to others, even apart from the issue of whether or not funding is 
attached. For example, a vice president at one college said the initiative was partly about 
“bragging rights” and argued that competition would exist even without the incentive funding: 

They [college presidents] are the most competitive 34 people you can have in the 
room. The biggest laughing point for me is when you hear that this isn’t about 
competition among the colleges. That sounds good but ... they do compete. If we 
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have a drop in [achievement] points, the president will be in here jumping up and 
down on my desk wanting to know why. 

Some interviewees expressed concern that colleges may be unfairly compared with one another 
without taking into account factors, such as differences in student characteristics, that might 
account for differences in institutional performance. A few college respondents, including one 
president, predicted that as colleges seek to improve their achievement point totals, they will be 
less likely to share effective practices with other institutions, particularly with those in their 
enrollment catchment areas. 

Creates potential for “gaming” of the system 

At almost half of the colleges, interviewees expressed the concern that colleges may be able to 
increase achievement points without really improving student success by “gaming” the system. 
Respondents said that some gaming already occurs under the current funding system. Pressure to 
game the system would increase if substantial funding were attached to the initiative. Those who 
raised this concern were reluctant to provide specific examples of how gaming might occur. 
However, several interviewees said that colleges could increase their achievement point totals by 
focusing on increasing student enrollment, rather than improving student progression and 
achievement. This would of course not further the goals of the initiative, and it might lead to 
increased competition for enrollments, particularly in areas where there are lots of postsecondary 
options. 

Could devolve into unhealthy competition for base-budget funding 

Several college presidents and senior administrators expressed concern about where the initiative 
funding would come from and how it would be allocated. In particular, they feared that the 
initiative may evolve in such a way that funding would come from colleges’ base budgets, a 
situation that would promote “unhealthy competition” as colleges would be forced to “win back” 
funds that could go to better performing colleges. The college leaders we interviewed were 
unanimous in their belief that to provide the right incentives to colleges, funding for the initiative 
must continue to involve new money above the base-budget, FTE-generated funds. One college 
trustee said that whether the initiative is “a lever or a hammer” depends on whether the 
legislature provides new funds for the initiative or uses it to allocate base funding. 

Creates uncertainty about future funding 

Several college leaders expressed concern that the initiative might not be adequately funded by 
the legislature in future years. Some cited the experience of Washington’s earlier, short-lived 
performance funding system (1998-99). Both supporters and critics of the current initiative 
expressed concern and skepticism about the stability of its future funding. 

Other presidents and college leaders warned that support for the initiative may be threatened 
because it did not come out of the legislature and as a result lacks legislative champions. One 
long-serving college president worried that the college achievement data may be viewed as “self- 
serving,” since the measures were not designed by educators: 
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My fear with this one is that I don’t see it as direct response to any request from 
the legislature. So how successful we will be in using this to create that 
confidence among legislators in what we do — I just don’t know where this will 
go. 

Does not make up for unfunded FTEs 

While the leaders we interviewed welcomed the opportunity to earn additional funds to use 
toward student success activities, even those with a favorable view of the initiative indicated that 
they would prefer “fully-funded FTEs” to the incentive system model. They argued that FTEs 
were currently not fully funded by the state, so any additional funding the legislature was willing 
to spend on community and technical college education should be allocated to FTE funding. 



C. Views on the Amount of Funding Needed to Encourage Change 

Some college personnel we interviewed felt that, in instituting the Student Achievement 
Initiative, the State Board overlooked the substantial efforts already taking place at their 
institutions to improve student success. Faculty and student services staff in particular stressed 
their personal and professional commitment as educators working at community colleges to 
helping students succeed. A few said they were offended by the implication that they would be 
more motivated to help students if their colleges were rewarded financially. 

Still, many respondents, including most college leaders, acknowledged that colleges would 
change the way they operate if given sufficient incentives. Every college leader we interviewed 
said that the $500,000 to be allocated to colleges for performance during the current academic 
year (2008-09) is not enough to attract much attention to the initiative, let alone provide 
incentive to alter institutional behavior. However, most of the college presidents we interviewed 
indicated that the $7 million that the SBCTC has requested for the initiative in its biennial budget 
for 2009-1 1 ($3. 5 million per year, or approximately $100,000 per college) would provide 
sufficient incentive for colleges to make substantial changes in policy and practice in order to 
improve their performance, particularly since these funds would be added to colleges’ base- 
budget funding. Again, college leaders were adamant that the allocation of new funds, rather 
than the reallocation of base budget funds, was essential for the long-term sustainability of the 
initiative. 
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Chapter 7 

Higher Education Policymakers’ Views of the Initiative 



To date, efforts to promote the Student Achievement Initiative have been primarily targeted at 
stakeholders within the community college system. Ultimately, however, the goal is to secure 
ongoing state appropriations for performance funding. This will require political support from 
such external players as the Higher Education Coordinating (HEC) Board, the governor’s office, 
and of course, the legislature itself. To get some perspective on the initiative’s prospects for 
support among state policy makers, we interviewed several of them, all of whom were fa mi liar 
with the initiative. These included staff from the governor’s office, a key legislator, and board 
and staff members from the HEC Board, as well as SBCTC system trustees and executive staff. 



A. Views of the Initiative among External State Policy Makers 

The state-level policy makers we interviewed were generally enthusiastic about it. An analyst 
from the Government Management Accountability and Performance (GMAP) office said that 
Governor Chris Gregoire is closely following the initiative. 

We have all the measures of this Student Achievement Initiative on our 
performance measure dashboard. We will be looking at those measures and 
work[ing] with the State Board for insight as to why certain results are occurring. 

The governor’s interest is focused on what is happening to students. The funding 
is really a bonus from our point of view. The idea of focusing on real outcomes 
for people is different than how we typically operate programs in government. 

A legislator who has followed the initiative observed: “With ideas like the Student Achievement 
Initiative out of the State Board, even without the details, I wanted to do it. It gets at what I’m 
trying to push, which is, how to measure what we want with the outcome we want [and] how do 
you provide rewards with additional flexibility.” 

A member of the Higher Education Coordinating Board praised the power of an accountability 
model that uses research-based indicators of student progress. This HEC Board member was 
particularly pleased that the initiative encourages colleges to reflect on their own data and 
improve in relation to their own historical performance, taking into account their distinctive mix 
of students, goals and strengths. “You get out of the competitive T’m better than the college 
down the road’ mode. ... It’s done in a way that institutions improve their own statistics and 
don’t get into the excuses game. I think it has great potential and power!” The board member 
argued that the initiative’s approach to funding represents a significant improvement over 
traditional ETE-based budgeting because it provides incentives to colleges not only to get 
students in the door but to ensure that they leave with a credential. 

Beyond the few state-level policy makers who are closely following the development of the 
initiative, the level of awareness of the Student Achievement Initiative among state policy 
makers is limited, according to persons we interviewed. The SBCTC has just begun to educate 
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legislators beyond a few with an interest in community and technical colleges, and the initiative 
currently lacks a strong legislative advocate. 



A staff member from the HEC Board observed that one of the challenges for the initiative is that 
once one gets beyond a broad understanding of its principles and overall design, the details of the 
achievement points and how the funding will be allocated are complicated. This person and other 
respondents encouraged the State Board to develop a succinct explanation of the initiative’s 
mechanics or accept that it is too complicated for most audiences and simply provide reassurance 
that the details are available to those who really want them. 



B. State Board Views on the Initiative 

There seems to be a strong sense of unity among State Board members about the direction they 
have set for the initiative. In interviews. State Board members described their commitment to 
create clearer and more seamless pathways through college programs to increase student success. 
They discussed how the Student Achievement Initiative in particular, while acknowledging the 
importance of student access to college, emphasizes increasing rates of completion among 
students who have enrolled. It thus serves to increase colleges’ focus on outcomes, not just 
enrollments. As one member observed, “We’ve got a nice new elegant measurement system and 
... for the first time in this state we are going to pay for some results, as opposed to putting 
money on the table at the front end.” 

Another board member echoed those comments, saying: 

If we can switch from FTE [enrollment funding] to achievement, I think that we 
as an organization are better served to represent to taxpayers and the legislature 
and local communities that people going through are getting valuable skills that 
will enhance our community and the tax base. I think I would be much more 
comfortable saying we have 10% fewer ETEs, but a 20% increase in students 
meeting their goals. That to me is a stronger argument than “we have more butts 
in seats, but aren’t sure about outcomes.” 

A senior executive from the State Board office observed that there has been a significant shift on 
the part of the State Board in terms of how it views its role compared to the past. “The Board has 
made it clear that they want to be at the front end of the policy process, not just voting up and 
down at the end.” While the Board is conscious of the potential for pushback, particularly from 
college leaders and faculty groups, and they acknowledge that change in large bureaucratic 
systems is threatening. But they seem resolute in their commitment to the initiative. As one 
board member observed. 

If you don’t live on the edge, you’re taking up too much room! .... If we are as 
good as we say we are, we should go out and prove it. We sell people this mantra 
that we are good . . . that we change students’ lives, that we can help the economy. 

If we can’t, then we should know it. 
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C. Funding the Initiative in a Challenging Fiscal Environment 



According to officials at the Higher Education Coordinating Board, SBCTC Executive Director 
Charlie Earl has been very strategic in introducing the new initiative, being careful not to get too 
far ahead of his board or the college presidents. A year ago he laid the groundwork for this year’s 
budget request by presenting information about the initiative in meetings with the HEC Board, 
the governor, her cabinet, and key legislative leaders. As mentioned, the State Board is 
requesting $7 million for the biennium to support the initiative. 

Now, however, projections of a multi-billion dollar shortfall in state revenue have made the 
prospects for funding of the initiative uncertain. An analyst in the governor’s office was 
guardedly optimistic about the initiative’s chances of getting a positive reception from the 
legislature, despite the challenging fiscal environment. He suggested that the initiative 
demonstrates the State Board’s commitment to accountability for student outcomes, which will 
benefit the community and technical college system as it competes with other state agencies for 
limited resources. 

My personal opinion is that these types of initiatives are more important in 
recessions and in times of tight money, because they demonstrate commitment to 
performance beyond money... The governor and I suspect that the legislature will 
look much more favorably on a system that is documenting its performance than 
one that is saying, “trust us.” 

Other state policy makers, however, echoed concerns we heard at several of the colleges about 
future funding of the initiative. A legislator who is familiar with the initiative raised questions 
about its prospects for support in light of the downturn in the economy and projected budget 
deficits. This legislator argued that the lack of a strong base of legislative champions does not 
bode well for the State Board’s biennial budget request for the initiative. 

The possibility of a re-allocation of the colleges’ base budgets was a particular concern of a few 
state policy makers. One state policy maker and long-time observer of the community and 
technical college system said that local boards and college presidents are very protective of their 
budgets and face significant pressures from faculty groups and local communities to secure a 
stable stream of income for their colleges. If, because of the shortfalls, base budgets were 
reallocated to fund the new initiative, he predicted that it would increase the likelihood of 
resistance. The only solution to prevent this from happening, in his opinion, would be to reduce 
the pot of incentive money. He acknowledged, however, that this might well mean that the 
system would be less likely to get the desired behavior changes at the colleges. “You need to get 
$2 or $2.5 million [per year] to get people’s attention.” He went on to observe that while the 
relatively long development period was important for creating buy-in from the colleges, the 
downside is that the Board is seeking continuing funding for the initiative in a time when there is 
no money. 

A staff member from the HEC Board observed, “If I were in [SBCTC Executive Director Charlie 
Earl] Charlie’s shoes, I would be very nervous right now. You take it to the legislature and the 
one outcome could be that they like it, and then say we are taking seven million out of your 
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current budget, which we may be cutting because of the fiscal environment, and use that seven 
million. That is every president’s worst nightmare about how this is going to work — that it will 
come out of their hide.” 

State Board members we interviewed said they and their colleagues remain firmly committed to 
the goals of the initiative, despite the uncertainty about funding. One State Board member was so 
enthusiastic about the initiative that he suggested that they proceed with it regardless of whether 
the legislature provides new funding. “Even if we get no money, this should be our priority. . . . If 
there is not an additional funding stream I would say that we take it out of the base of whatever it 
is we have.” This Board member argued that it is hard to imagine how either the State Board or 
professional educators could disagree with a focus aligned toward student achievement. “You 
don’t need more money to do that.” 

Yet, board members generally agreed that the absence of new legislative funding for the 
initiative would create challenges for the State Board as it proceeds with the initiative. One board 
member said: “Our goal has always been to go after new money for this as an incentive to the 
colleges .... I hate to lose it. It’s a big decision that we will have to make.” One senior Board staff 
person said that the colleges will be involved in the decision on how to proceed should the 
legislature not approve the funding request. “The Board will have a process. It will listen to 
everyone around the system and then make the decision as to whether or not to go forward. 
Commensurate with the process will be the developing of options and alternatives. It won’t be a 
ramrod.” 
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Chapter 8 
Conclusion 



Higher education performance funding policies hold the promise of spurring colleges and 
universities to adopt new policies and practices that lead to improved student outcomes. Such 
policies, it is hoped, will help to better align institutional efforts with state goals and lead to a 
better return on the public’s investment. Yet, despite high expectations, most higher education 
performance funding measures in the U.S. have been short-lived, undergone frequent changes, 
and involved relatively small amounts of money. In general, the expected benefits of 
performance funding have not been realized. 

By tying state funds directly to institutional performance, the Student Achievement Initiative is 
consistent with prior performance funding systems that have been implemented across the 
country. The initiative does, however, differ from performance funding policies in other states in 
at least one key respect: rather than just rewarding colleges for successful degree completion and 
other “final” outcomes, the Student Achievement Initiative rewards colleges for increasing the 
rate at which students, starting from remedial levels, progress incrementally toward completing 
credentials. 

Colleges support the initiative’s goals and the principles of the achievement point framework 

Among those we interviewed at 17 of the state’s 34 community and technical colleges, there was 
widespread support for the initiative’s goal of focusing attention and resources on helping all 
students, including those who start in basic skills and college remedial programs, to successfully 
progress through their educational programs. There was also strong support for the focus on 
increasing student progression across intermediate levels of achievement. The faculty, student 
support staff, and administrators we interviewed generally agreed that this incremental 
progression framework is much more relevant to their work with students than accountability 
measures based only on final outcomes. Virtually everyone who was familiar with the research 
the State Board staff conducted to inform the development of the overall framework and the 
specific performance measures had high regard for it. 

Colleges spent most of the learning year grappling with the performance data 

While college personnel generally supported the initiative’s student progression framework, 
there was considerable uncertainty about exactly how student progression is measured. Most 
colleges struggled throughout the learning year to understand the details of the achievement 
point performance measures. At the time of our interviews, most colleges were still in the 
process of understanding how their baseline year achievement point totals were calculated. The 
lack of institutional research capacity hampered several colleges’ ability to analyze the data for a 
more thorough understanding of their college’s performance. Even colleges with greater IR 
capacity were having difficulty making sense of the data and figuring out how to use the 
information to monitor student progression and evaluate efforts to improve student outcomes. 
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As a result, most colleges had not begun to use the Student Achievement Initiative database to 
develop strategies for increasing student success. Only a handful was planning new strategies to 
improve their achievement point performance. Most were relying instead on student success 
activities previously in place at their colleges. While college personnel were optimistic that these 
existing activities would help to increase student attainment of achievement points, most such 
efforts were small in scale. College leaders have not yet engaged faculty and staff to figure out 
what changes in policy and practices are needed to increase the rate at which students attain 
achievement points on a substantial scale. 

Colleges are anxious about how the achievement point framework will work in practice 

The lack of clarity about how exactly how the initiative measures student progression has led to 
anxiety among college personnel over how they will rank relative to other colleges. There is 
indeed considerable uncertainty about how the measurement framework will work in practice — 
whether, for example, it might disadvantage colleges that serve larger numbers of disadvantaged 
students, which are more expensive to serve, or whether it might disadvantage colleges that have 
traditionally performed well and may therefore have less room for improvement. 

Awareness of the initiative is limited among the colleges’ rank and file 

A greater level of understanding of the initiative and, in particular, of how college performance 
is measured across the student achievement categories will be required to bring about more 
broad-based involvement of faculty and staff in efforts to improve student achievement. While 
State Board staff did much to try to build awareness of the initiative among the colleges, at the 
colleges in this study familiarity with the initiative is mostly limited to college leadership, senior 
administrators, and a small number of college personnel involved in initiative efforts at the 
colleges. Several colleges were waiting for a better understanding of their performance on the 
complete 2007-08 data before expanding communication about the initiative to faculty and staff. 
Senior administrators at several colleges said that, having had to respond to numerous State 
Board initiatives in recent years, they are reluctant to engage campus- wide constituencies in 
another external directive until they are assured it is likely to gain support from the state 
legislature. 

The lack of awareness and clarity about the initiative should not be too surprising, however. The 
initiative was developed and is being implemented on a relatively short timeline, and the colleges 
were still in the first “learning year” of the initiative during our interviews. Still, college 
personnel at several colleges suggested that clear and transparent communication about the 
initiative would allay some of the uncertainty and anxiety surrounding the effort and help build 
support for it. 

Tying funding to performance raises the stakes 

The potential of significant funding tied to institutional performance has also generated anxiety 
across the colleges, particularly among presidents. Most presidents we interviewed seem to agree 
that the amount currently being sought by the State Board from the legislature — $7 million for 
the biennium, or about $100,000 per year per college — is large enough to motivate colleges to 
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make substantial changes in the way they operate, particularly because the funds would be added 
to their base budgets. Presidents we interviewed were unanimous in the view that to be effective 
in improving college performance, the initiative must bring new funding to the table over and 
above base budget funding. College leaders feared a situation where there would be no new 
funds to support performance funding and where colleges would instead be forced to earn some 
portion of their base budget funding based on the achievement point measures. Presidents argued 
that this would lead to unhealthy competition among the colleges and would probably kill 
support for the initiative from the colleges. Although there were exceptions, most state-level 
observers we interviewed seemed to agree that funding above and beyond the base budget 
funding is essential for the initiative to be effective and sustainable. 

Some state policymakers strongly support the initiative, but legislative support is uncertain 

The state-level stakeholders we interviewed are very enthusiastic about the initiative’s model of 
performance accountability and improvement. An analyst from the Governor’s office suggested 
that the initiative represents a refreshing effort on the part of educational institutions to document 
for policymakers, taxpayers, and the public how well they are serving students. There are strong 
synergies between the initiative and the governor’s push for more outcome-oriented and 
accountable state programs. Among the state-level policymakers we interviewed, there was a 
consensus that accountability pressures across state agencies are not likely to abate, and that the 
Student Achievement Initiative could give the community and technical colleges an advantage as 
they compete with other sectors for scare state resources. 

The Higher Education Coordinating Board seems committed to advocating with the legislature 
for the SBCTC’s request for new funding to support the initiative. However, state-level 
stakeholders indicated that the initiative is not well known among legislators and lacks strong 
champions in the legislature. 

Fiscal woes threaten state funding for the initiative 

The State Board’s request to the legislature is threatened by the looming fiscal crisis facing 
Washington State and the country more generally. A decision by the legislature not to fully fund 
the initiative could affect future college support. College leaders clearly oppose the idea of using 
the initiative’s performance measurement framework to determine base budget funding. At this 
point, however, the State Board seems intent on moving ahead with the request. Particularly 
given the state’s budget woes, it is essential that the Board develop a clear and compelling 
message and a strategic communications plan to educate policy makers about the initiative and 
its potential benefits for students and the state. 

While legislators may not support the initiative at the level requested by the State Board, they 
might be willing to provide more modest funding and allow the Board to extend the learning 
period for the initiative through the biennium as the state weathers the economic downturn. 

Given the fact that colleges are just beginning to analyze their data and consider changes in 
practices that would improve student achievement, extending the learning period would give the 
Board and the colleges a chance to deepen awareness and support of the initiative among faculty 
and staff, use the data to identify areas of weakness, and implement and evaluate strategies for 
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improving student achievement. This would also increase the opportunity to examine to what 
extent and in what ways colleges change their practices in response to the initiative. Finally, 
given the widespread interest in the initiative by other states and funders nationally, the State 
Board may well be able to raise private foundation funds to supplement state funding of the 
initiative during such a period of further experimentation and evaluation. 
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